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Abstract 

It has been shown that the Kullback-Leibler divergence is a Lyapunov 
function for the replicator equations at evolutionary stable states, or ESS. 
In this paper we extend the result to a more general class of game dy- 
namics. As a result, sufficient conditions can be given for the asymptotic 
stability of rest points for the entire class of incentive dynamics. The 
previous known results will be can be shown as corollaries to the main 
theorem. 

1 Information Theory and The Replicator Dy- 
namics 

Information theory was originally developed by Claude Shannon and Warren 
Weaver [ShaOl, SW49] as a mathematical framework to describe problems in 
communication including, but not limited to, data compression and storage. 
They introduced measures of information called entropy 1 . Shannon's entropy, 
denoted H(P), is a measure of the average uncertainty in a random variable, P. 
It can be interpreted as the average number of bits needed to encode a message 
drawn i.i.d. from P. Maximizing the entropy can be used to give a lower bound 
on this average number of bits needed for encryption. 

For our purposes, the concepts of cross entropy and relative entropy will be 
of great use. The Kullback-Leibler divergence (KL divergence or Dxl) [KL51], 
or relative entropy is a measure of information gain (loss) from one state to 
another. More precisely, it is an average measure of the additional bits needed 

1 ln fact, the Shannon entropy is simply the Boltzmann entropy [Jay65] without the con- 
stants 



to store y given a code optimized to store x. It is defined as 
Dkl{x\\v) = y^Xa In — 

a. a 

= H(x) - H(x,y). 

where H(x, y) is the cross entropy of x and y. It should be clear that minimizing 
Dkl with respect to y is equivalent to minimizing the cross entropy term as well. 
Intuitively, this is trying to find the best distribution to approximate the 'true' 
distribution x and is well known as the Principle of Minimum Discrimination 
or Minimum Discrimination Information. 

Recall the definition of an evolutionary stable state or ESS [S + 74]. 

Definition 1. A strategy profile x <E A is an ESS if and only if u(x, x) > u(x, x) 
for every x ^ x in a neighborhood of x. 

In this context there is a single population playing a symmetric game against 
itself. It has been shown by Weibull [Wei97], and Harper [Harll] that the 
KL divergence is a Lyapunov function for the replicator equation at an ESS 2 . 
Further connection between evolutionary games and information theory can be 
realized by expanding the KL divergence in a Taylor series along x = y and 
noting that the Hessian term is positive definite and is thus a metric. The 
derived metric, a localization of the global divergence, is called the Shahshahani 
metric [Sha79] and it has been shown that the replicator dynamics are gradient 
flows of this metric [HS98]. 

2 Incentive Stable States 

In [Fryl2], the notion of an incentive for a game was developed. Furthermore, 
a family of game dynamics was derived from these incentives and shown to be 
fully general in the following sense: any valid game dynamic can be achieved 
from an appropriate choice of incentive. To recap: an incentive, ip(x), is valid 
if and only if, for every i and a, xi a = =>■ ipi a (x) > and J2 a fia( x ) 7^ ~ 1- 
The corresponding incentive dynamic is then given by 

%ia = <Pia{x) - x ia y^^tpip(x). 



The deep connections between information theory and the replicator dynam- 
ics lead us to believe that some of these properties are more general. Unfortu- 
nately, most of our incentive dynamics are not gradient flows of some Rieman- 
nian metric, but the Principle of Minimum Discrimination is compelling enough 
for us to believe we may be able to describe asymptotically stable states for the 

2 This result continues to be true for n-population games. 
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incentive dynamics. We begin by defining a notion of incentive stability that is 
closely related to the notion of ESS. 

Definition 2. A strategy profile x is an incentive stable state or ISS if and only 
if 

Xi ■ < Xi ■ , V^ 

Xi Xi 

for x ^ x in a neighborhood of i. 

The interpretation is exactly the same as in the ESS case: x is preferred to 
all distributions sufficiently close. 

We can now show that all ISS are asymptotically stable for the corresponding 
incentive dynamics. Note: if there is only one agent we have a necessary and 
sufficient condition for the Kullback-Liebler divergence to be a strict Lyapunov 
function. 

Theorem 1. If the state x is an interior incentive stable state for the corre- 
sponding incentive dynamics, then DfCL(£i\ is a local Lyapunov function. 

Proof. Define Vi(x) — fjfi(»i||a;<) and V(x) — J2i^i( x )- Then we have the 
following: 



Vi(x) = - y^xjq 

a 



P 

— tPiai?) < 



(Pia(x) - X ia ^ <Pip( x ) 


X, 



□ 



3 Examples 

The replicator dynamics can be achieved using the incentive 

<Pia(x) = X ia (fia(x) +9i(x)), 

where / is the fitness landscape, and g is an arbitrary function. Thus the ISS 
condition trivially reduces to the familiar ESS condition. 

There are however, many other dynamics that are asymptotically stable at 
ESS. For example, Nagurney and Zhang [NZ96], Lahkar and Sandholm[LSOcS], 
and Harper [Harll] all show independently that the euclidean distance can be 
used as a Lyapunov function to establish this fact for the projection dynamics. 
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3.1 Best Reply 

The Best Reply dynamic as denned by Young [YouOl] has a simple incentive 
function, tp ia (x) — BR ia (x), where BR ia (x) = 1 if ei a is a best reply to the 
current state x. A tiebreaker is assumed for instances where there is more than 
one best reply. Thus the incentive dynamic is 

X^ a B R^ a (x^ -Eia- 

The ISS condition is rather simple to interpret: 

Xi ' *C Xi 



Y,BR ia {x)<Y, 



la. 



X ' R 

1 < — — , where is the best reply to x 
x if} < x i/3 
This occurs trivially at an ESS. 
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